Some computer programs, called application programs (applications), are designed to help users perform a certain type of work. An example of an application is an index card program, which helps users store and retrieve small volumes of textual information. The index card program maintains an electronic version of an index card file.
A user interacts with an application by inputting commands and data to the computer, which are received by the application. An example of commands that might be available in an index card program are add card and delete card, which a user might input in order to add or delete an index card. To add an index card to the file, a user would choose the add card command and input the text that is to be stored on an index card.
A user typically inputs a command by making a series of choices. These choices are often arranged in a command hierarchy. The command hierarchy represents all of the combinations of choices that a user can make in order to input a command. The application moves through, or "traverses," the command hierarchy in response to the user's choices. When the application reaches the bottom of the command hierarchy, the user has inputted a complete command. The application then executes a set of instructions that is associated with that command.
FIG. 1 is a diagram of the command hierarchy for the index card program. The command hierarchy has a root node 101, first-level nodes 111-118, and second level nodes 121-125. The first-level nodes are said to be child nodes of the root node, and the second-level nodes are said to be child nodes of a first-level node. When traversing the command hierarchy, the application begins at the root node. By making a choice, the user may move the application to any child of the root node, like the card node 114. From the first-level node, the user continues to move the application to successive children until the application reaches a node without any children, such as the delete node 122 or the duplicate node 123. At this point, the user has inputted a complete command, which the application will execute. For example, by moving the application first to the card node and then to the duplicate node, the user can input the duplicate card command.
Applications often present command hierarchies visually as drop-down menu structures. FIGS. 2A-2C are display images of the visual interface presented by the index card program. FIG. 2A is a display image of the visual interface presented by the index card program when the user is at the root node of the command hierarchy. The application's visual interface is presented in an application window 201. The application window contains a menu bar 202 and a client area 203. The menu bar in turn contains menu bar items 211-216. Each of the menu bar items corresponds to a first-level node in the command hierarchy. At this point, the user can either input data in the client area or choose a menu bar item.
If the user chooses a menu bar entry, the application moves to the corresponding first-level node in the command hierarchy. FIG. 2B is a display image of the visual interface presented by the index card program after the user has chosen the card menu bar item. The card menu bar item is highlighted, and a drop-down menu 220 is displayed beneath it. The drop-down menu contains drop-down menu items 221-224. These correspond to second-level nodes 121-124. At this point, the user can choose a drop-down menu item.
When the user chooses a drop-down menu item, the application moves to the corresponding second-level node in the command hierarchy. FIG. 2c is a display image of the visual interface presented by the index card program as the user is selecting the duplicate drop-down menu item. If that node has no children, then the user has inputted a complete command, and the application executes it. In FIGS. 2A-2C, the user has inputted the duplicate card command, and the application will duplicate an address card.
The application may also display choices in other screen objects besides menu bars and pull down menus, such as dialog boxes. A dialog box is a region of the screen containing a free-form arrangement of choice controls. FIG. 3 is a screen image showing a dialog box for finding a card that contains certain text. The Find dialog box 301 contains text box 302, check box 303, option buttons 304 and 305, and command buttons 306 and 307. Each dialog box control corresponds to a choice a user can make.
A user usually reaches a dialog box by selecting a pull-down menu item that ends in an ellipsis. FIG. 4 is a screen image that shows how a user would reach the find dialog box. The user has chosen the search menu bar item 401. The user proceeds to choose the find . . . pull-down menu item 402. This causes the display of the Find dialog box 301.
Some choices corresponding to nodes of an application's command hierarchy are not displayed in any screen object. Examples include the next card node 117, the previous card node 118, and the escape node 125. The user must read the application's documentation in order to find out, first, that these choices exist and second, how to make these choices.
Applications typically allow users to make the choices necessary to input a command by either typing or using a pointing device, such as a mouse. While both choice methods are useful, applications would be more convenient if a user could also make choices and otherwise input commands by speaking words.
A computer can "hear" and recognize spoken words using a voice recognition system. In a voice recognition system, a microphone converts an audio signal to an analog electrical signal. A digitizer then converts the analog electrical signal to a digital electrical signal. A voice recognition program compares the digital electrical signal to phonetic models for different words in order to discern which word the user has spoken.
Dedicated facilities are available that allow a user to make choices and otherwise input commands for applications by speaking words. These facilities use a voice recognition system, and act as a front-end to applications. The facilities receive recognized words from the voice recognition program, translate them into commands that the current application can understand, and submit those commands to the current application. Typically, these facilities depend upon the current application's prompts for valid commands to inform the user which words are valid to speak at a given time. That is, these facilities present no independent indication of which spoken words they will recognize at a given time.
This omission poses problems. First, while these facilities allow a user to input commands that do not appear in the drop-down menu structure by speaking them, there is no indication that these commands are available for voice input. Second, in order to choose menu items that are phonetically similar to other menu items, these systems sometimes require users to speak words different from those appearing in the menu item. Because there is no visual indication of these substitutions, the facilities require users to memorize the substitute words.